Skip to content

ChangeFeedProcessor: Fixes first-change skip during initial startup by anchoring StartTime#5617

Merged
kirankumarkolli merged 14 commits into
masterfrom
users/nalutripician/fix-changefeedprocessor-startasync-5268
Apr 8, 2026
Merged

ChangeFeedProcessor: Fixes first-change skip during initial startup by anchoring StartTime#5617
kirankumarkolli merged 14 commits into
masterfrom
users/nalutripician/fix-changefeedprocessor-startasync-5268

Conversation

@NaluTripician
Copy link
Copy Markdown
Contributor

@NaluTripician NaluTripician commented Feb 18, 2026

Summary

Fixes issue #5268 where ChangeFeedProcessor can skip the first change when started without explicit start options.

Bug Description

When ChangeFeedProcessor is started with default options (no WithStartTime, no continuation, no start-from-beginning), it uses the default start-from-now path.

This path anchors at the first service read (IfNoneMatch: *) rather than at StartAsync() call time. Because startup includes asynchronous lease acquisition and load balancing, there is a window where documents can be written after StartAsync() returns but before the first read reaches the backend. Those documents can be skipped.

Root Cause

  • PartitionLoadBalancerCore.Start() schedules background work and returns immediately.
  • The first change feed read happens later after lease operations.
  • Default Now behavior is based on first read timing, not StartAsync() timing.
  • Documents created in that gap may be missed.

Fix

In ChangeFeedProcessorCore.StartAsync():

  • On first initialization only, if no explicit start option is configured:
    • StartFromBeginning == false
    • StartTime == null
    • StartContinuation is null/empty
  • set changeFeedProcessorOptions.StartTime = DateTime.UtcNow before async initialization.

This anchors start behavior at processor startup time and closes the timing gap.

Why This Is Safe

  • Does not override explicit user start settings.
  • Applies only on first startup when no lease continuation exists.
  • Existing continuation/checkpoint behavior remains unchanged.

Tests Added

Unit tests

Microsoft.Azure.Cosmos.Tests/ChangeFeed/ChangeFeedProcessorCoreTests.cs

  • StartAsync_SetsStartTime_WhenNoStartOptionsProvided
  • StartAsync_DoesNotOverrideExplicitStartTime
  • StartAsync_DoesNotSetStartTime_WhenStartFromBeginning

Emulator regression test

Microsoft.Azure.Cosmos.EmulatorTests/ChangeFeed/DynamicTests.cs

  • TestWithRunningProcessor_ImmediateWriteAfterStart
  • Starts processor and immediately writes documents without setup delay.

Validation

A/B test on the new emulator regression

  • With fix enabled: 10/10 passed
  • With fix temporarily removed: 7/10 failed, showing the original issue reproduces with high probability
  • Fix restored and revalidated: stable pass behavior

Additional checks

  • Targeted unit tests pass.
  • No diagnostics errors in changed files.

Files Changed

  • Microsoft.Azure.Cosmos/src/ChangeFeedProcessor/ChangeFeedProcessorCore.cs
  • Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.Tests/ChangeFeed/ChangeFeedProcessorCoreTests.cs
  • Microsoft.Azure.Cosmos/tests/Microsoft.Azure.Cosmos.EmulatorTests/ChangeFeed/DynamicTests.cs

@NaluTripician
Fix ChangeFeedProcessor missing first change on startup
8fa0028 
Addresses #5268 by setting an implicit StartTime during first StartAsync when no explicit start options are set.

Also adds unit coverage for StartTime option behavior and an emulator regression test for immediate writes after StartAsync.
github-actions[bot]
github-actions Bot reviewed Feb 18, 2026
View reviewed changes
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good!

@NaluTripician NaluTripician changed the title ChangeFeedProcessor: prevent first-change skip during startup (#5268) ChangeFeedProcessor: Fixes first-change skip during startup Feb 18, 2026
@NaluTripician NaluTripician self-assigned this Feb 18, 2026
@NaluTripician NaluTripician added the auto-merge Enables automation to merge PRs label Feb 18, 2026
NaluTripician

This comment was marked as outdated.

Sign in to view

@kirankumarkolli
Copy link
Copy Markdown
Member

kirankumarkolli commented Mar 4, 2026

Please update Title with initialization state also

NaluTripician and others added 2 commits March 4, 2026 11:36
@NaluTripician
Merge branch 'master' into users/nalutripician/fix-changefeedprocesso…
676016c 
…r-startasync-5268
@NaluTripician
Test: Use >= instead of == for processedDocCount check
a4dec3f 
Address review feedback to handle case where more than 10 docs
could be processed in the callback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NaluTripician
NaluTripician commented Mar 4, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor Author

@NaluTripician NaluTripician left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: DynamicTests.cs line 116 — @kirankumarkolli

Good catch — the == 10 check is fragile. If the callback fires with a batch that crosses 10 (e.g., count goes from 8→12), the ManualResetEvent would never be set and the test would time out. Changed to >= 10 to be defensive. Pushed in a4dec3f.

@NaluTripician NaluTripician changed the title ChangeFeedProcessor: Fixes first-change skip during startup ChangeFeedProcessor: Fixes first-change skip during initial startup by anchoring StartTime Mar 4, 2026
@NaluTripician
Copy link
Copy Markdown
Contributor Author

NaluTripician commented Mar 4, 2026

Updated the PR title to include initialization state context as requested.

This was referenced May 1, 2026
Merged
Open
Closed
Closed
Merged
Open
ananth7592 added a commit that referenced this pull request May 4, 2026
@ananth7592
ChangeFeed: Fixes PREVIEW guard on GetChangeFeedProcessorBuilderWithA…
36fb41a 
…llVersionsAndDeletes

Moves the abstract method declaration out of the #if PREVIEW block
so it compiles into the GA (non-preview) build output.

Added a check inside ChangeFeedProcessorCore that skips appending startTime fix done as part of #5617

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ananth7592 added a commit that referenced this pull request May 4, 2026
@ananth7592
ChangeFeed: Fixes PREVIEW guard on GetChangeFeedProcessorBuilderWithA…
19e5c57 
…llVersionsAndDeletes

Moves the abstract method declaration out of the #if PREVIEW block
so it compiles into the GA (non-preview) build output.

Added a check inside ChangeFeedProcessorCore that skips appending startTime fix done as part of #5617

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ananth7592 added a commit that referenced this pull request May 4, 2026
@ananth7592
ChangeFeed: Fixes PREVIEW guard on GetChangeFeedProcessorBuilderWithA…
f5fc04f 
…llVersionsAndDeletes

Moves the abstract method declaration out of the #if PREVIEW block
so it compiles into the GA (non-preview) build output.

Added a check inside ChangeFeedProcessorCore that skips appending startTime fix done as part of #5617

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced May 4, 2026
Merged
Merged
This was referenced May 7, 2026
Open
Open
@dependabot dependabot Bot mentioned this pull request May 11, 2026
Closed
@NaluTripician NaluTripician mentioned this pull request May 11, 2026
Open
ananth7592 added a commit that referenced this pull request May 12, 2026
@ananth7592
ChangeFeed: Fixes PREVIEW guard on GetChangeFeedProcessorBuilderWithA…
392c077 
…llVersionsAndDeletes

Moves the abstract method declaration out of the #if PREVIEW block
so it compiles into the GA (non-preview) build output.

Added a check inside ChangeFeedProcessorCore that skips appending startTime fix done as part of #5617

Contract changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ananth7592 added a commit that referenced this pull request May 12, 2026
@ananth7592
ChangeFeed: Fixes PREVIEW guard on GetChangeFeedProcessorBuilderWithA…
c0b56b1 
…llVersionsAndDeletes

Moves the abstract method declaration out of the #if PREVIEW block
so it compiles into the GA (non-preview) build output.

Added a check inside ChangeFeedProcessorCore that skips appending startTime fix done as part of #5617

Contract changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@NaluTripician NaluTripician mentioned this pull request May 12, 2026
Merged
@dependabot dependabot Bot mentioned this pull request May 12, 2026
Closed
ananth7592 added a commit that referenced this pull request May 14, 2026
@ananth7592
ChangeFeed: Fixes PREVIEW guard on GetChangeFeedProcessorBuilderWithA…
9a76d37 
…llVersionsAndDeletes

Moves the abstract method declaration out of the #if PREVIEW block
so it compiles into the GA (non-preview) build output.

Added a check inside ChangeFeedProcessorCore that skips appending startTime fix done as part of #5617

Contract changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced May 15, 2026
Merged
Closed
Open
Open
Merged
Open
Closed
Open
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@github-actions github-actions[bot] github-actions[bot] left review comments

@kirankumarkolli kirankumarkolli kirankumarkolli approved these changes

@Praveen-Msft Praveen-Msft Praveen-Msft approved these changes

@khdang khdang Awaiting requested review from khdang

@sboshra sboshra Awaiting requested review from sboshra

@adityasa adityasa Awaiting requested review from adityasa

@neildsh neildsh Awaiting requested review from neildsh

@FabianMeiswinkel FabianMeiswinkel Awaiting requested review from FabianMeiswinkel

@kirillg kirillg Awaiting requested review from kirillg

@Pilchie Pilchie Awaiting requested review from Pilchie

Assignees

@NaluTripician NaluTripician

Labels

auto-merge Enables automation to merge PRs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@NaluTripician @kirankumarkolli @Praveen-Msft